{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Linear Regression: L1 vs L2\n",
    "-------------------------------\n",
    "\n",
    "This function shows how to use TensorFlow to solve linear regression via the matrix inverse.\n",
    "\n",
    "It is important to know the effect of loss functions in algorithm convergence. Here we will illustrate how the L1 and L2 loss functions affect convergence in linear regression.  We will use the same iris data set as in the prior recipe, but we will change our loss functions and learning rates to see how convergence changes.\n",
    "\n",
    "<img src=\"../images/04_L1_L2_learningrates.png\" width=\"512\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We start by loading the necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from sklearn import datasets\n",
    "from tensorflow.python.framework import ops\n",
    "ops.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# L1-Loss\n",
    "--------------\n",
    "\n",
    "Here, we will illustrate linear regression with the L1-Loss. Later in this script, we will illustrate the same problem with L2-Loss.\n",
    "\n",
    "The equation for the L1 Loss for Linear Least Squares is\n",
    "\n",
    "$$S = \\sum_{i=1}^{N}\\left| y_{i} - \\hat{y_{i}} \\right|$$\n",
    "\n",
    "Where $N$ is the number of data points, $y_{i}$ is the i-th actual y-values, $\\hat{y_{i}}$ is the predicted i-th y-value.\n",
    "\n",
    "We start a computational graph session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now we load the Iris data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# iris.data = [(Sepal Length, Sepal Width, Petal Length, Petal Width)]\n",
    "iris = datasets.load_iris()\n",
    "x_vals = np.array([x[3] for x in iris.data])\n",
    "y_vals = np.array([y[0] for y in iris.data])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Set some model parameters.\n",
    "\n",
    "An important parameter to take note of is the learning rate.  If the learning rate is too large, the model will not converge.  If the learning rate is too small the model will converge too slowly.\n",
    "\n",
    "Here are two learning rate values to show convergence and non-convergence.\n",
    "\n",
    "Convergence happens below 0.35, try setting the learning rate less than that for convergence.  To illustrate non-convergence, set the learning rate to 0.4 or higher."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "batch_size = 25\n",
    "learning_rate = 0.1 # Will not converge with learning rate at 0.4\n",
    "iterations = 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now we can initialize placeholders, model variables, and model operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Initialize placeholders\n",
    "x_data = tf.placeholder(shape=[None, 1], dtype=tf.float32)\n",
    "y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)\n",
    "\n",
    "# Create variables for linear regression\n",
    "A = tf.Variable(tf.random_normal(shape=[1,1]))\n",
    "b = tf.Variable(tf.random_normal(shape=[1,1]))\n",
    "\n",
    "# Declare model operations\n",
    "model_output = tf.add(tf.matmul(x_data, A), b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Next, we declare the l1-loss function and the optimization function.  After that we initialize the model variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Declare loss functions\n",
    "loss_l1 = tf.reduce_mean(tf.abs(y_target - model_output))\n",
    "\n",
    "# Declare optimizers\n",
    "my_opt_l1 = tf.train.GradientDescentOptimizer(learning_rate)\n",
    "train_step_l1 = my_opt_l1.minimize(loss_l1)\n",
    "\n",
    "# Initialize variables\n",
    "init = tf.global_variables_initializer()\n",
    "sess.run(init)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now we start the training loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step #25 A = [[ 1.95419633]] b = [[ 3.00462866]]\n",
      "Step #50 A = [[ 1.62139654]] b = [[ 3.64862919]]\n"
     ]
    }
   ],
   "source": [
    "# Training loop\n",
    "loss_vec_l1 = []\n",
    "for i in range(iterations):\n",
    "    rand_index = np.random.choice(len(x_vals), size=batch_size)\n",
    "    rand_x = np.transpose([x_vals[rand_index]])\n",
    "    rand_y = np.transpose([y_vals[rand_index]])\n",
    "    sess.run(train_step_l1, feed_dict={x_data: rand_x, y_target: rand_y})\n",
    "    temp_loss_l1 = sess.run(loss_l1, feed_dict={x_data: rand_x, y_target: rand_y})\n",
    "    loss_vec_l1.append(temp_loss_l1)\n",
    "    if (i+1)%25==0:\n",
    "        print('Step #' + str(i+1) + ' A = ' + str(sess.run(A)) + ' b = ' + str(sess.run(b)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# L2-Loss\n",
    "--------\n",
    "\n",
    "Here, we will illustrate linear regression with the L2-Loss..\n",
    "\n",
    "The equation for the L2 Loss for Linear Least Squares is\n",
    "\n",
    "$$S = \\sum_{i=1}^{N}\\left( y_{i} - \\hat{y_{i}} \\right)^{2}$$\n",
    "\n",
    "Where $N$ is the number of data points, $y_{i}$ is the i-th actual y-values, $\\hat{y_{i}}$ is the predicted i-th y-value.\n",
    "\n",
    "We start a computational graph session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# L2 Loss\n",
    "# Reinitialize graph\n",
    "ops.reset_default_graph()\n",
    "\n",
    "# Create graph\n",
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Same as before, we initialize the placeholders, variables, and model operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Initialize placeholders\n",
    "x_data = tf.placeholder(shape=[None, 1], dtype=tf.float32)\n",
    "y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)\n",
    "\n",
    "# Create variables for linear regression\n",
    "A = tf.Variable(tf.random_normal(shape=[1,1]))\n",
    "b = tf.Variable(tf.random_normal(shape=[1,1]))\n",
    "\n",
    "# Declare model operations\n",
    "model_output = tf.add(tf.matmul(x_data, A), b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Here is the loss function, variable initialization, and optimization functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Declare loss functions\n",
    "loss_l2 = tf.reduce_mean(tf.square(y_target - model_output))\n",
    "\n",
    "# Declare optimizers\n",
    "my_opt_l2 = tf.train.GradientDescentOptimizer(learning_rate)\n",
    "train_step_l2 = my_opt_l2.minimize(loss_l2)\n",
    "\n",
    "# Initialize variables\n",
    "init = tf.global_variables_initializer()\n",
    "sess.run(init)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now we can start our linear regression training with the L2 function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step #25 A = [[ 1.39285481]] b = [[ 3.81929326]]\n",
      "Step #50 A = [[ 1.1075846]] b = [[ 4.47525787]]\n"
     ]
    }
   ],
   "source": [
    "loss_vec_l2 = []\n",
    "for i in range(iterations):\n",
    "    rand_index = np.random.choice(len(x_vals), size=batch_size)\n",
    "    rand_x = np.transpose([x_vals[rand_index]])\n",
    "    rand_y = np.transpose([y_vals[rand_index]])\n",
    "    sess.run(train_step_l2, feed_dict={x_data: rand_x, y_target: rand_y})\n",
    "    temp_loss_l2 = sess.run(loss_l2, feed_dict={x_data: rand_x, y_target: rand_y})\n",
    "    loss_vec_l2.append(temp_loss_l2)\n",
    "    if (i+1)%25==0:\n",
    "        print('Step #' + str(i+1) + ' A = ' + str(sess.run(A)) + ' b = ' + str(sess.run(b)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Here is matplotlib code to plot the loss for the L1 and L2 loss functions applied to the same linear regression problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEZCAYAAAB1mUk3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VGX2wPHvCR0TQCAU6cgqooC4gEgNsAoqIE1URBH8\nKa5LU1CwLWHVVVzXhuzK6i5YQKRItdCjKKAgSC+CAoKAgVACQhKS8/vj3mAI6cnMnZmcz/PcJ5Nb\nz9xJ7pn7vvd9X1FVjDHGFG5hXgdgjDHGe5YMjDHGWDIwxhhjycAYYwyWDIwxxmDJwBhjDJYMjI+J\nSH8RWeF1HMY7IvKpiNzjdRwma5YMQoiI/CQiHTKYX0xEZrjLU0SkrZ9Dy7Axi4jUcuO56O9QRO4V\nkbUickJE9onIuIzWS7N+iojULcigA4WINBWR+SIS506bReRZESnrdWzpicgYEXkv7TxVvUVV3/cq\nJpMzlgwKjxXA3cBBrwNJJ7NWj6WAYUAF4HqgIzAyD/sJGiJSJIN5LYHlOJ/flapaHugMnAMaex2f\nCR2WDAoBVU1S1TdUdSWQkt36InKfiGwVkZMisktEHkyzrJ2I/Cwij4rIYRE5ICL3pVleXkTmud/o\nVwOX5zHmiar6taqeU9WDwBSgVVZhZ/JeRESeFpE9InJIRCaLSBl3WQkReV9EjojIMRH5RkQi05yD\n3e452C0id2Wy/zHuXdc0d921ItIozfKqIjJTRH519zMkg23fF5HjQP8MDjEO+K+qvqSqse652a+q\nY1X1yzT7Guh+ZkdF5DMRqZlmWYqIDBKRne7yN9O9h+y2fVhEdgI73XmvuXdrJ0RkjYi0dud3Ap4E\n7hCReBFZ785fLiIDc/B5pN4p3isie91z9mRG5934gKraFCIT8BPQIZt1fgbaZrPOzUBt93Ub4DRw\nrft7OyAJGAMUcdc9DZR1l09zp5LA1cB+4MtMjlMLSAbCcvDeZgN/z2J5ClA3g/kDcS5itYDSwCzg\nXXfZg8BcoAROMmkChLvrnQDquetVBq7K5LhjgASgh3s+RgA/uq8FWAs85f5eG9gF3Jhu267u7yXS\n7bs0zh1Adp9Xd/c9XoHzBe9J4Ot052YeEAHUAH4FbsrFtguBsqnxAX2Bcu76j+DcbRZP857eSxff\ncmBgFp/He2n+HlKAiUBxoBFwFueOyPP/r1CfPA/ApgL8MAsoGWSwzWxgiPu6Hc7FPyzN8sNAc/fi\nkAj8Ic2y58lnMgAGAPuA8lmsk1kyWAI8lOb3K9wLcJi736+Ahum2KQ3EuRf4ktnENgZYmeZ3AQ7g\n3MU0B/akW380zjf91G1jsth3Nfd9XZFm3jjgGHAKeNKd9ykwIM06Ye5nVCPNubkhzfKPgMdzsW27\nbM5BXOo5zEEyyOjzSHSPm/r3UDXN8m+APl79TxWmyYqJzEVE5GYRWeUWGxzD+fZfMc0qR1U1bXHT\nbzjfqCNxvgHvT7Nsbz5j6Q78HeisqnF52MVl6WLYCxTD+bb/Ps633mkisl9EXhSRIqr6G3AH8Gfg\noFt5e2UWx/g59YU6V7AD7nFrAdXSVPweA54AKmW0bQaO4VyMq6bZ/yhVvRQnQRd1Z9cCXk89DnAU\npw6lWpp9HU7zOvXzyum2aT9PRGSEW6x0zH1PZbjw7yMrGX0eRXE+j+xiNT5kycBcQESKAzOBl4BI\n98LzGZmUyacTi/PNrkaaeTUzWTcnsXTGKTLooqpb87ibX3AueKlq4RRzHVanPuJZVb0aaAl0Be4F\nUNXFqnoTUAXYAbydxTHOv18REaC6e9yfgR9Vtbw7XaqqZVW1a5ptM634dpPSN0DPbN7jz8CgdMcJ\nV9XV2WyX023Px+jWDzwO9HbXvRQ4ye9/H9lV5Gf6eeQgVuNDlgxCT3G3YjR1KgLORV5ESrrrlBCR\nEplt705HVDVFRG4GbsrJgd27hVlAtIiUEpEGZFwpmpYAJdPFLOI8IvsB0EtVv8vJ8XHfV5opDPgQ\neEREaotIOE6x1TT3vUWJyDXueqdwLkrJIlJJRLqKSGl33imcsvvM/FFEurvn+hGccu7VwLfASRF5\nXERKikgREblaRJrm8P2Ac+Ed6O4jtXK7OlAnzTpvAU+65xsRKSsivXO4/9xuG4FzTo66f1N/deel\nOgzUdpNiRjL9PNzlOfnSYXzAkkHo+QTn1vqM+3OMO38HTlnwZcDnwG9pnxpJpaqngKHADLfY4E6c\nStaspP02OATn4nAQ+J87ZbdtfLqY2wNP4xQ/fOo+mXJSRD7JZj+b0+3nPlX9L05x0JfAbnf+UHeb\nKjh3QSeALThl2x/g/F+MwCnuOQK0BR7O4thzcYqVjuE8vttDVZPdC1xX4Fqc+pxfce4wymRzTn5/\nU6pfAx1w6mp2uJ/Jp26s49115gAv4hR3HQc24jx+mvbckNHvedh2Ic7fz073Pf3GhUVdM3Au6EdF\nZG0G+/gfmX8eWcZqfEucIk4f7Vzkv0AXnFvyRu68l3D+QRJw/hgGqOpJnwVhjA+JyBjgclW91+tY\njMkPX98ZTAI6pZu3CLhaVa8FfsCpUDPGGOMhnyYDVf0K59Y57bwlacoHV+NUthljjPFQ0exX8amB\nOA2UjAlKqjrW6xiMKQieVSCLyFNAkqpO9SoGY4wxDk/uDESkP3ALzlMSWa1nTxIYY0weqGquHtP1\nx52BkObZYbch0eNAN1VNyG5jr5toB8o0ZswYz2MIlMnOhZ0LOxdZT3nh02QgIlOBlcAVbi+HA3Ce\njQ4HFovIOhH5ly9jMMYYkz2fFhOpat8MZk/y5TGNMcbknrVADhJRUVFehxAw7Fz8zs7F7+xc5I9P\nWyDnl4hoIMdnjDGBSETQXFYge93OwBgTwmrXrs3evfnqxdxkoVatWuzZs6dA9mV3BsYYn3G/oXod\nRsjK7Pzm5c7A6gyMMcZYMjDGGGPJwBhjDJYMjDHGYMnAGFNI1alTh2XLll00Pykpidtvv506deoQ\nFhbGl19+meV+2rdvz//+l92AfoHPkoExxqTTpk0bpkyZQtWqVb0OxW8sGRhjTBrFihVj6NChtGzZ\nkrCw/F0i582bxzXXXEP58uXp0KED27dvP79s3LhxVK9enTJlynDVVVexfPlyANasWUOzZs0oW7Ys\nVatWZeTIkfmKIacsGRhjjA/s3LmTvn378sYbbxAbG8vNN99M165dOXfuHDt37mTChAl89913nDx5\nkoULF1K7dm0Ahg0bxvDhwzlx4gS7d++mT58+fonXkoExxjMiUiBTIJo+fTpdunShQ4cOFClShJEj\nR3LmzBlWrlxJkSJFSExMZPPmzZw7d46aNWtSp04dAIoXL86uXbs4evQopUuXpnnz5n6J15KBMcYz\nXvbf72u//PILtWrVOv+7iFCjRg0OHDjA5ZdfzmuvvUZ0dDSVK1emb9++HDx4EID//ve/7Nixg/r1\n63P99dfzySef+CVeSwbGGOMDl1122UX9Mv38889Uq1YNgDvvvJMVK1acX2f06NEAXH755UydOpXY\n2Fgef/xxevfuzZkzZ3weryUDY0yhlZiYSEJCwvkpOTn5/PyzZ88CnF+WlaSkpAv2c+7cOfr06cMn\nn3zC8uXLOXfuHC+//DIlS5akZcuW7Ny5k+XLl5OYmEjx4sUpVaoURYoUAWDKlCkcOXIEgLJlyyIi\n55f5lNfDs2Vz66fGmOAVyP/DtWvX1rCwMA0LC1MR0bCwMH3mmWcuWpY67d27N8P9REVFXbTuPffc\no6qqc+bM0QYNGmi5cuU0KipKt27dqqqqGzdu1ObNm2uZMmW0QoUK2rVrVz148KCqqvbr108rVaqk\nERERes011+i8efMyfQ+ZnV93fq6ut9ZrqTHGZ6zXUt+yXkuNMcYUKEsGxhhjgiAZ2C2mMcb4XOAn\nA7dG3xhjjO8EfjI4fdrrCIwxJuRZMjDGGGPJwBhjTDAkg3LlvI7AGGNCnjU6M8b4jDU68y1rdGaM\nMfmU2bCX33zzDTfddBMVKlSgcuXK3HHHHRw6dCjT/diwlzkgIv8VkcMisjHNvEtFZJGI7BCRhSJS\n1pcxGGNMbhw7doxBgwaxd+9e9u7dS3h4OAMGDPA6LJ/z9Z3BJKBTunmjgSWqeiWwDHjCxzEYY0yO\nde7cmV69ehEeHk7JkiUZPHgwK1euzNO+bNhLl6p+BRxLN/s24F339btAd1/GYIwx+fHFF19w9dVX\n53o7G/Yye5VU9TCAqh4CIrNcO48Z2RgTBKKjQeTiKTo65+tntm4B2LhxI88++ywvv/xyrre1YS8L\n2uLFXkdgjPGV6Gin/7H0U1bJIKfr5tOuXbu45ZZbGD9+PC1btsz19sE27GVRvxzlQodFpLKqHhaR\nKsCvWa0cvWjR+c7qoqKiiIqK8kOIxpjCbO/evdx4442MGTOGvn375mkfl112GZs3b75gXvphL++8\n805OnTrFgw8+yOjRo3n33XfPD3sJMGvWLHr37k1cXBylSpXK9FgxMTHExMTkKc5U/kgG4k6p5gH3\nAeOA/sDcrDaOvvZan94GGmMKr9RhL1MVLVqUw4cP07FjRwYPHswDDzyQo/2kDnuZqkiRIvTp04dx\n48axfPly2rRpw2uvvXbBsJcHDhygVatW54e9TG0vMGXKFDp16kTFihVzPOxl+i/KY8eOzcVZcOV2\naLTcTMBU4BcgAdgHDAAuBZYAO4DFQLkstle9775Mh3wzxgQ2gnDYy7Fjx2pYWJhGRERoRESEhoeH\na0RERKb7sWEv/UBEVG+/HaZP9zoUY0weWAtk3ypcLZC7dvU6AmOMCXmBf2cQwPEZY7Jmdwa+Vbju\nDIwxxvicJQNjjDGWDIwxxlgyMMYYgzctkHPnhRfgCevY1JhgVKtWLURyVY9pciFtdxf5FfhPE4nA\nuXMQZjcxxhiTE6H5NFHJknDmjNdRGGNMSAv8ZHDJJXD6tNdRGGNMSLNkYIwxJkiSwW+/eR2FMcaE\ntMBPBn/5C1x6qddRGGNMSAv8p4kCOD5jjAlEofk0kTHGGJ+zZGCMMcaSgTHGGEsGxhhjCIZksGwZ\nfPml11EYY0xIC/yO6lasgORkaNvW60iMMSZkBf6dQenS1ujMGGN8LPCTgXVHYYwxPmfJwBhjTJAk\nAysmMsYYnwr8CuSGDSEpyesojDEmpFnfRMYYE2KsbyJjjDF5YsnAGGOMJQNjjDEeJgMReURENovI\nRhGZIiLFvYrFGGMKO0+SgYhcBgwBrlPVRjhPNd2Z4crJyTB2rB+jM8aYwsfLYqIiwCUiUhQoDfyS\n4VphYfC3vzlJwRhjjE94kgxU9Rfgn8A+4ABwXFWXZLiyiPVPZIwxPuZJozMRKQfcBtQCTgAzRaSv\nqk5Nv250dDSkpMDYsUR16UJUVJR/gzXGmAAXExNDTExMvvbhSaMzEekNdFLVB9zf7wGuV9XB6dZz\nGp3VrQuLF8Pll/s9VmOMCTbB1OhsH9BCREqKiAAdgW2Zrm39ExljjE95VWfwLTATWA9sAAT4T6Yb\nDB0KFSr4JzhjjCmErG8iY4wJMcFUTGSMMSaAWDIwxhhjycAYY4wlA2OMMQRLMli6FJYv9zoKY4wJ\nWcGRDFaudBKCMcYYnwiOZHDJJXD6tNdRGGNMyAqeZGAtkI0xxmeCIxmULm13BsYY40PBkQysmMgY\nY3wqOJJBw4ZwZ8YDoRljjMk/65vIGGNCjPVNZIwxJk8sGRhjjLFkYIwxJgfJQEQuF5ES7usoERnq\njmFsjDEmROTkzmAWkCwi9XBGI6sBXDRwvU+lpMBTT/n1kMYYU5hk+zSRiKxT1etE5DHgrKqOF5H1\nqtrE58GlPk2kCsWKwZkzzk9jjDGZ8tXTREkichfQH1jgzvPvFVnEGp4ZY4wP5SQZDABuAJ5X1Z9E\npA7wgW/DyoD1T2SMMT5TNLsVVHUrMBRARC4FIlT1RV8HdhHrn8gYY3wmJ08TxYhIGREpD6wD3haR\nV3wfWjpWTGSMMT6Tk2Kisqp6EugJvKeq1wN/8m1YGXj0UahUye+HNcaYwiDbYiKgqIhUBfoA3j3f\n2b+/Z4c2xphQl5M7g78BC4HdqrpGROoCP/g2LGOMMf5kvZYaY0yI8Uk7AxGpLiKzReRXETksIrNE\npHrewzTGGBNoclJMNAmYB1wGVAPmu/OMMcaEiJwkg0hVnaSq59xpMhCZ3wOLSFkRmSEi20Rki4hc\nn+UGixc7kzHGmAKXk2RwRET6iUgRd+oHHC2AY78OfKqqVwGNgW1Zrv3tt7B8eQEc1hhjTHo5SQYD\ncR4rPQQcBHrjdFGRZyISAbRR1UkA7h3HySw3skZnxhjjM9kmA1Xdp6rdVDVSVSupanecBmj5URfn\njmOSiKwTkf+ISKkstyhd2vomMsYYH8lJo7OMPAq8ls/jXgf8RVXXishrwGhgTPoVo6OjnRcbNxJ1\n4gRR+TioMcaEopiYGGJiYvK1jzy1MxCRn1W1Rp4PKlIZWKWqdd3fWwOjVLVruvV+b2cwZw5MmgRz\n5+b1sMYYUyj4ajyDjOSrJZiqHgZ+FpEr3Fkdga1ZbtSwIdxzT34Oa4wxJhOZ3hmISDwZX/QFKKWq\neS1iSt1/Y+AdnIFyfgQGqOqJdOtYC2RjjMmlvNwZWHcUxhgTYvxZTGSMMSaEBHwySEhI8DoEY4wJ\neQGfDP7yl79gRUXGGONbAZ8MVq9ezVtvvQWqMHKk89MYY0yByms7g02q2tAH8aQ/jv7www+0atWK\nWbNm0bpDB4iPhxIlfH1oY4wJWnmpQM708VARyazLCQGq5OYg+VGvXj0mT55Mnz592F+6NGGnT1sy\nMMaYApZVW4GPgClk3NagpG/CydjNN9/M4MGDORIdTdm4OEqUL+/PwxtjTMjLqtHZd0B/Vd2cwbJ8\ndUeR4+DStDNQVQ6WKcOETp14bsYMRHJ1B2SMMYVGQbczGA5k1q10j9wcpCCICJXr1mXn+vVOhbIx\nxpgCk9cK5OGqmp9eS3N6nAtbIE+Zwk916tCiRw/at29Pr169uPnmmwkPD/d1KMYYEzT81h2FiOxT\n1Zq53jD3x8mwO4rY2Fhmz57NrFmzWL16NR06dKBXr1507dqVsmXL+josY4wJaP5MBn6vM8hMXFwc\n8+fPZ9asWcTExNCxY0eef/55GjRo4OvwjDEmIAVNF9YFqXz58vTv35958+Zx4MAB2rVrR7t27Rgy\nZAhxcXFeh2eMMUEh02QgIvEicjKDKR64zI8x5lhERATDhw9n27ZtpKSkUL9+fcaPH09SUpLXoRlj\nTEAL6S6sN2/ezCOPPMKBAwd49dVX6dSpUwFGZ4wxgSn0xzNYuBBSUuDmm3O8D1VlwYIFjBgxgk6d\nOvHGG29YGwVjTEgL/fEM1q2DL77I1SYiQteuXfnuu++IiYlh/PjxPgrOGGOCV76GrvS7Sy6BX37J\n06YRERHMnz+fG264gSuuuILOnTsXcHDGGBO8guvO4JJL4PTpPG9eu3ZtZsyYwb333su2bdsKMDBj\njAluwZUMSpeG337L1y5at27NSy+9RNeuXTl69GgBBWaMMcEtuJJBPu8MUt1333307NmT3r1722On\nxhhDsD1NtGcPrF8PPfLfT15ycjLdu3fnsssu46233rInjIwxISP0Hy0tYPHx8bRs2ZIHHniAoUOH\n+uw4xhjjT5YM8mDPnj3ccMMNTJ06lfbt2/v0WMYY4w+h387AB2rXrs3kyZO59957rULZGFNoFfo7\ng1SPPvooe/fuZebMmVZ/YIwJanZnkA8vvPACu3fv5p133vE6FGOM8bvgSgaqMGyY87OAlShRgg8/\n/JAnn3yS7du3F/j+jTEmkHlaTCQiYcBaYL+qdstg+cXFRCVLwrFjUKqUT2KaOHEiEydOZNWqVZQo\nUcInxzDGGF8KxmKiYcDWXG1RQA3PMvPggw9Sq1Ytnn76aZ8dwxhjAo1nyUBEqgO3ALkrpPdxMhAR\n3nnnHaZNm8bixYt9dhxjjAkkXt4ZvAo8Rm6H0CyA/omyU6FCBSZPnsyAAQOIjY316bGMMSYQeNKF\ntYjcChxW1e9FJArItGwrOjr6/OuoqCiifHxnkKpjx47cfffd3HXXXYwbN47rrrvOHjk1xgSkmJgY\nYmJi8rUPTyqQReTvQD/gHFAKiAA+VtV70613cQXyjBnQujVUrerzOBMTE/nb3/7GRx99RGJiIj17\n9qRXr17ccMMNFClSxOfHN8aYvAjK7ihEpB0wIsdPE3lAVdmyZQsff/wxH3/8MYcOHaJ79+50796d\nqKgoSpYs6XWIxhhzniUDP9m9ezcff/wx8+bNY+PGjXTo0IEuXbpwyy23UNUPdyzGGJOVoEwGWQnU\nZJDW0aNH+fzzz1mwYAELFy6kbt26dOnShSFDhlChQgWvwzPGFEKWDDyWlJTEypUrefvtt9m3bx9L\nliyhePHiXodljClkLBkEiJSUFHr27ElkZCT/+c9/7CkkY4xfBWML5Nz7/HOYP9/rKLIUFhbG+++/\nz6pVq5gwYYLX4RhjTLY8aWeQLxs2wJEj0LWr15FkKSIignnz5tGyZUuuuuoqOnbs6HVIxhiTqeC7\nM7jkEp+3QC4odevW5cMPP6Rv377s3r3b63CMMSZTwZkM/NACuaC0b9+e6OhounXrxsmTJ70Oxxhj\nMhR8yaB06aBKBgB//vOfadu2LXfffTfJycleh2OMMRcJvmQQRMVEab3++uvEx8fz5JNPEoxPSBlj\nQlvwJYNGjWDQIK+jyLXixYszc+ZMlixZQqNGjZg0aRIJCQleh2WMMYC1M/A7VWXJkiW8/PLLbNq0\nicGDB/PQQw9Rvnx5r0MzxoSIwtHOIMiJCDfeeCMLFy7k888/Z+fOndSrV48hQ4awZ88er8MzxhRS\nlgw81KhRIyZPnszmzZsJDw+nefPmfPzxx16HZYwphKyYKICsXbuWXr16cffdd/Pss8/amAnGmDyx\nvolCQGxsLH369KFEiRJMnTrV6hKMMblWeOoMHn4YUlK8jsInIiMjWbx4MVdffTXNmjVjw4YNXodk\njCkEgvPOoHRpiI112hyEsKlTpzJs2DDeeOMN7rrrLq/DMcYEicJTTBQZCVu2QKVK/g/KzzZs2ECP\nHj3o1q0bL730ko2PYIzJVuEpJrrkEoiP9zoKv2jcuDFr165l9+7dREVFsX//fq9DMsaEoOBMBi1a\nwCefeB2F35QvX565c+fSrVs3mjZtyqJFi7wOyRgTYoKzmGjFCvi//4Nt2yAsOPNZXsXExNC3b18G\nDRrE008/bY+fGmMuUnjqDFTh66+hVSsohENKHjx4kDvvvJOSJUsyZcoUKlas6HVIxpgAUnjqDESg\ndetCmQgAqlatytKlS2nSpAlNmjRhwoQJxBeSOhRjjG8EZzIwFC1alBdffJFp06axfPlyatWqxdCh\nQ9mxY4fXoRljgpAlgyDXqlUrZs6cyYYNGyhTpgxt27blpptuYv78+TaQjjEmx4KzzsBkKiEhgRkz\nZjB+/HgSExNZvnw55cqV8zosY4wfFZ4K5LQOH4ZDh6BxY/8EFUSGDBnCjh07+OSTTyhWrJjX4Rhj\n/KTwVCCntXp1UI585g+vvvoqRYsWZciQITbUpjEmS8GfDLp0gYMH4bvvvI4k4BQtWpRp06axcuVK\nXn31Va/DMcYEME+SgYhUF5FlIrJVRDaJyNA876xIEfjzn2HChAKMMHSUKVOGBQsW8M9//pO5c+d6\nHY4xJkB5UmcgIlWAKqr6vYiEA98Bt6nq9nTr5awCOTYWrrgCdu2CChV8EnOwW7NmDbfccgsLFy7k\nuuuu8zocY4wPBU2dgaoeUtXv3dengG1AtTzvMDISunaFSZMKKMLQ06xZMyZOnMhtt91mnd0ZYy7i\n+dNEIlIbiAGucRND2mU5f7T0hx8gMRGuvrqAIwwtL730Eh9++CErVqwgPDzc63CMMT4QdI+WukVE\nMcCzqnpRgba1Myh4qsqgQYPYtm0bc+bMoYIVqxkTcvKSDIr6KpjsiEhRYCbwfkaJIFV0dPT511FR\nUURFRfk8tlAmIrz11ls88cQTtGzZkk8++YR69ep5HZYxJh9iYmKIiYnJ1z48uzMQkfeAI6r6aBbr\n2J2BD02cOJHo6GhmzZpFy5YtvQ7HGFNAgqaYSERaAV8CmwB1pydV9fN061ky8LHPPvuM/v37M2HC\nBG6//XavwzHGFICgSQY5ledk8NNPzsA3t9xS8EGFoA0bNtClSxeGDBnCY489hhTSrsGNCRVB82ip\nz506BffdB0ePeh1JUGjcuDGrVq1i6tSpPPTQQ5w9e9brkIwxfhaayaBhQ7j9dhgzxutIgkb16tVZ\nsWIFx44d4/LLL+f111/nzJkzXodljPGT0EwGAH/7G0yfDps2eR1J0IiIiGD69OksWLCAmJgY6tat\nyyuvvMLp06e9Ds0Y42OhWWeQ6s03YfZsWLKk0A6RmR8bNmzgueeeY8WKFYwYMYIHH3yQ2NhYtm/f\nzvbt29mxY8f5140aNWL27NmUKVPG67CNKfSsAjm9c+egZUt4/3248sqCC6yQ2bx5M8899xyzZs2i\nevXq1K9fnyuvvJL69etTv359rrjiCp577jnWrFnDwoULKV++vNchG1OoWTLISHKy07Opybfk5GSK\nZHIuVZVRo0bx+eefs3jxYipXruzn6IwxqYKqBbLfWCIoMJklAnD++MaNG0dERARt2rRh6dKl1KhR\nw4/RGWPyI3QrkAPRL7/A449DXJzXkfiEiPDMM8/w0EMP0bZtW3bt2uV1SMaYHAr9O4P0tm+HG2+E\ncuWcqWxZKF8eHngA2rTx7bHXroV166BDB6dSu2JF3x7PI48++ijh4eFERUWxaNEiGjRo4HVIxphs\nhH6dQXpJSc439BMn4PhxZ/r5Z9i7F156KfvtT52CjLp+zmndhCo89RQsWOAkhEqVcv8egsSUKVMY\nOXIkc+fOpXnz5l6HY0yhYRXIvnT6NDz9NKxYAWvWXPio6jffwIMPwsSJ0KJF9vtShehomDEDli2D\nKlUyXzchAd57D3r2zP8obqpOMouIyN9+cmHBggUMHDjQ+j4yxo+sOwpfiYmBRo2c4TU///ziNgvN\nm8Po0dCnOQ0rAAAUW0lEQVSjBzz8sHO38dNPme9PBMaOheHDoVixjNc5d84Zue3KK2HVKrj00vy9\nh6NHoU8f55jpJSU5x/OBLl26sGjRIkaMGMHzzz9PwCR3Y8yFVDVgJyc8D61fr9qli2q1aqrz5mW/\nflyc6qBBquXLq9arp3r2bO6PmZysOm2a6pVXqrZrp/rVV7nfR3qLFjnv4ZFHVM+cuXj5Rx+pNmig\n+umn+T9WJn755Rdt2rSp9uvXT89mcV7i4uL09ddf13Hjxum5c+d8Fk9hlZiYqL/++qvXYRgfc6+d\nubve5nYDf06eJ4O9e1Vffln12LHcbbd1a+63SfWvf6k2bepcwFNS8raPVGfOqA4frlq9uurixZmv\nl5LiJLsrrlC96SYnGfnggnH69Gnt3bu3tmrV6oILUkpKiq5cuVL79++vZcuW1TvuuEPbt2+vnTt3\n1ri4uAKPIxAlJyf7/Bjbtm3Tpk2baqlSpbR///66c+dOnx/TeMOSQShITMx/Ekj19tuqvXurHj2a\n82O//bZq166qZcqobt5cMHGkkZycrE899ZTWrVtXV61apW+++aY2bNhQ69Wrpy+99NL5JJGUlKTD\nhw/XP/zhD7ply5YCjyMrKSkpunXrVp04caJOmTJFv/76az1w4IBPLtjx8fE6YsQILVasmNaoUUNv\nuukmHTZsmL711lv6xRdfFMi3+OTkZH399de1QoUK+q9//Uvj4uJ07NixWrFiRe3Xr59u3769AN6J\nCSSWDAqLTZtUH3jAKVLKSkpK3hNLYmL2+8+Hd999V8uVK6e33367LlmyJNML7eTJkzUyMlLnzJnj\ns1hUVQ8dOqQffPCB9u/fX6tVq6Y1a9bUe++9V++44w69/vrrtXLlylqiRAn9wx/+oH/605/0scce\n0+PHj+frmPPnz9datWrpPffcowcPHtRdu3bp/Pnz9R//+IcOHDhQb7jhBi1Xrpxef/31unfv3jwd\nY9++fdqxY0dt0aLFRXcCJ06c0Oeee04jIyO1b9++unXr1ny9HxM4LBkUFmfOOPUJQ4cW3F1ETm3Y\n4Nw9+PG433zzjVavXl2jo6Nz9e387NmzOnPmTL311lu1TJkyWqVKFa1Xr542btxYW7VqpZ06ddKe\nPXtqo0aNtFy5ctq9e3edMGGC7ty5U1MyeH+nT5/Wbdu26eeff64PPvig1qhRQxcuXJjr93PgwAHt\n3bu31qtXTxdnVXynzl3Kyy+/rJUrV8523fTbffDBBxoZGanPPfecJiUlZbruyZMn9YUXXtBKlSrp\nnXfeqdu2bcvxMRYvXqxPPfWUfvHFF1bHE0AsGRQmx46pNmyo+sQTqv37q773nn+Ou3Wr6rXXqnbr\npnr4sH+OqaoHDx7Uli1bavfu3XXjxo168uTJTNddt26dDhkyRCtWrKhRUVH67rvvamxsrP7yyy+6\nc+dOXbduna5YsUI/++wznTFjhq5cuTLLi2VmFi1apDVq1NBBgwZlGU+qc+fO6ZtvvqkVK1bUZ555\nRs9kVJmfiWXLlmmVKlX0hRdeyDBRpbVhwwbt1auXNmjQQL/77rscHyM+Pl5feOEFjYyM1H79+mVa\np5CUlKQffvihNmnSRBs0aKAjR47Ua6+9ViMjI3XgwIE6f/78XL03X9qyZYs++eST+o9//EM/+ugj\nXbVqVZ6L/BISEnTJkiU6duxY3bFjhw+iLTiWDAqb/ftVmzVT/etfVfNZZJErCQmqo0erVqmiOneu\n3w579uxZHT58uNavX19LlSql5cuX1yZNmmiPHj30kUce0ejoaG3cuLHWrFlT//rXv+ru3bt9HtPx\n48d1wIABWqdOHV2+fPlFy+Pj4/XTTz/VESNG6FVXXaVt2rTJc3HMvn37tHnz5tqjRw89ceLEBcuS\nkpJ0xowZ2rZtW61WrZo+//zzeb4gnzhxQp999lmtWLGi3nfffefP4+nTp3XChAlat25dbd26tc6f\nP/+Ci+qPP/6or7zyirZt21bLlCmjvXv31hkzZmhCQkKe4siP1atX62233aaVKlXSxx9/XB955BHt\n1auXNmvWTCtXrqzFixfXOnXqaMeOHXXEiBH6/vvv66ZNmzQxMfGC/cTFxemUKVP0jjvuOF9k9/DD\nD2uFChV06NCheuTIEb+/t5zISzKwRmcm7776Cu69F267DV599eLl69fDF19At25Qt26BHlpViY2N\nZe/evezZs4e9e/fy66+/0qlTJ9q3b09YmH+b0CxYsIBBgwbRu3dvevXqxfLly1m6dCnr1q2jadOm\ndOzYkY4dO9KiRYt8xZaQkMCwYcOIiYlh9uzZREZG8vbbb/Pvf/+bWrVqMWTIEHr06EGxzNqv5MLx\n48d57bXXGD9+PO3atePrr7+mRYsWjBo1ipYtW2a5bWxsLPPmzeODDz5gy5Yt3HPPPdx///1Zdk0S\nHx/P6tWrWbduHcePH+fUqVPEx8cTHx9//nVERARNmzalefPmNG/enKpVq57fXlVZunQpL7zwArt2\n7WLkyJHcf//9lC5d+qJjnT17lv3797Nz5042bNjA999/z/fff8/PP/9MgwYNaNSoEXv27GHt2rW0\nb9+ebt26ceutt1LFbSAaGxtLdHQ006dPZ/To0QwePJgSJUpccAxVZdOmTcyZM4e5c+eSkJBAs2bN\nzk+NGjW6aJuCYi2Qjf+dPAkbNmTcr9OSJc5ocx9/DE88AcOGQdHQ7Q4rLi6O4cOHs2XLlvMX/9at\nW3PJJZcU+LEmTZrEY489RnJyMj179mTw4ME0adKkwI8DzvuaOnUqHTp0yFM/U7t27eJ///sfkydP\npnbt2tx///3ccccdnD59mq+++ooVK1bw1VdfsX37dq677jqaN29O+fLliYiIIDw8nIiIiPOvjx07\nxpo1a/j222/59ttvKVWqFM2bN6dx48YsWLCAU6dOMWrUKPr27Uvx4sVzHeupU6fYtGkTGzZs4LLL\nLuNPf/pThskk1bZt2xg1ahRbtmzhxRdfpEePHqxcuZI5c+YwZ84cALp3785tt91GREQEa9asOT/t\n2rWLBg0a8Mc//pEKFSpQokSJ81PJkiXPvy5WrBjFihWjePHiF7yuXr06NWvWzDAuSwYmMO3eDYMG\nOS2z33kHrr3W64hCwp49ewgPD6dikHR4eO7cOT777DPeeecdli5dSrFixWjVqhWtW7emTZs2NG3a\nNFfflFWVn376iW+//ZZ169bRokULunfv7ve7QoBly5YxcuRIduzYwZVXXkn37t3p3r07DRs2RDIZ\nZfG3335j/fr1rFu3jhMnTpCQkEBCQgJnz549/zohIYGkpCSSkpJITEw8/zopKYk+ffowPKMeBbBk\nYAKZKrz7Lkye7PTH5ME/7AXOncv/XUpSkrMPG1I1106ePEl4eLgnF25fSUlJ4ciRI1QKgM4nLRmY\nwKea8cVz6VLYtcvpTfbECYiPhzNnnL6Urr764vXfeMPpb6l6dahRw/lZsybkZAzmqVPh+eed7sTT\nfhM9fBhuv93pRLBDh+z3c8UV8MMPULy4M5Uo4fxctQpq1bp4/f/8B3791SlaSztNnpxxd+bvvefs\ns1EjuOqq7OMxxmUjnZnAl9m36LVrneKksmWdqVIlKFXKeZ2RGjWcZLBqldP76/79sG+f06lg06YZ\nb3PihNOR4Lp1TkJIXyRRqRIMGeKMbVG/Pjz3nJMg6tVzpvR27oSUFOcOITHR6WE2MTHzbskPH3bW\nqVjRqVAvU8aZMqtT2LjReU+PPALXXAMjRsBNNwXPnUhcnHNubAjUoGB3BiZ0JCY6Y0pkNK7EihVw\nzz1w663wj39AFpWCJCTAv/8NL77oJIG//x3atvVd3NlJSIBp0+Cf/3TufhYsyN/+Tp92nvBq0MC5\n47jqKud1pUoFk2hSUpy6oaefdpJBx47wwQdQsmT+9+0La9bAuHHOl4Abb/RNEWZcnPPlplSpgt93\nBqyYyJiMHDrkXOzeew+6dPE6mrxThQMHnCKxnK6vevHFLSHBeeR361bYtu33n9WqOU+GpffTT9Cp\nk/MI8Z//nPVjwtu2wf/9n3Pct96COnXgs8+c7tMD1cmTzp3i2287d5v33w8DBmR+nu+91xkM67rr\nfp+uvDLjOqjkZGf5jz86d4FjxsDAgT5/qs6SgTGZSUnxvtLal0aNcu4Y6tSB2rWdC8/s2fDKK3Dz\nzTnbx8mTGde5JCU5CWPKFGeMjWbN4C9/gc6dL74LW7fO+ab9wAPZn++ffnKKzq66KvPiwII0fTp8\n+qlTR5OZdeucpDBtGowfD/36XbzOkSNOG5r16531161zRk/8+mto3Pji9XfsgMsvd9YfNcpZ99VX\nc/a5nDgBmzdDq1Y5fpsQZMlARDoDr+EMsPNfVR2XwTqWDIzJifh45+K6Z48zxcY6F5sbbijYOoYz\nZ5yL6oQJzuh7o0fnfV+zZzsV+du3O8kgtcjq7rszHjFw+nSYORPOnnWmM2ecn4MGOXcj6f32m1Ms\nEx8PQ4fCypXOHUBmdUppJSQ45y2nbRVOnnSOlV1jP1VYtMiJ+7bbLl62fbszcmLq9MMPTtHVrFm5\n+hyDJhmISBiwE+gI/AKsAe5U1e3p1rNk4IqJiSEqKsrrMAKCnYvfeXoukpKyv/jlREqK8wBAanFV\ngwZOsVR633/vfMsuVcqpf0j9WaMGVKly8bkYOdIZirZECSdxvfJKxuOXB4rkZGjSxHlY4PrrnREU\nmzTJU11LMD1N1Bz4QVX3AojINOA2YHuWWxVidgH8nZ2L33l6LgoiEYBTnFSzpjN17pz5etdem2WD\nxYvOxcsvO5XYR45k/DRYoClSxHmCzCNeJYNqwM9pft+PkyCMMabglCvnTCZbXtWoZXT7YuVBxhjj\nEa/qDFoA0ara2f19NE6Xq+PSrWcJwhhj8iBYKpCLADtwKpAPAt8Cd6nqNr8HY4wxxps6A1VNFpHB\nwCJ+f7TUEoExxngkoBudGWOM8Y+AbJIpIp1FZLuI7BSRUV7H428i8l8ROSwiG9PMu1REFonIDhFZ\nKCJ+aLLpLRGpLiLLRGSriGwSkaHu/MJ4LkqIyDcist49F2Pc+bVFZLV7Lj4UkULT+aSIhInIOhGZ\n5/5eKM+FiOwRkQ3u38a37rxc/48EXDJwG6S9CXQCrgbuEpH63kbld5Nw3n9ao4ElqnolsAx4wu9R\n+d854FFVbQDcAPzF/VsodOdCVROA9qraBLgWuFlErgfGAf90z8Vx4H4Pw/S3YcDWNL8X1nORAkSp\nahNVTX1EP9f/IwGXDEjTIE1Vk4DUBmmFhqp+BRxLN/s24F339btAd78G5QFVPaSq37uvTwHbgOoU\nwnMBoKq/uS9L4NT3KdAemOXOfxfo4UFofici1YFbgHfSzO5AITwXOI/qp7+W5/p/JBCTQUYN0qp5\nFEsgqaSqh8G5SAKRHsfjVyJSG+cb8WqgcmE8F26xyHrgELAY2A0cV9UUd5X9wGVexednrwKP4bZP\nEpEKwLFCei4UWCgia0QktZOmXP+PBGKZmjVIMxcQkXBgJjBMVU8V1vYn7oWuiYiUAWYDGQ1/FvLn\nRkRuBQ6r6vciEpU6m4uvHSF/LlwtVfWQiEQCi0RkB3l474F4Z7AfqJnm9+o4ndkVdodFpDKAiFQB\nfvU4Hr9wKwFnAu+r6lx3dqE8F6lU9STwBdACKOfWs0Hh+V9pBXQTkR+BD3GKh14DyhbCc5H6zR9V\njQXm4BS15/p/JBCTwRqgnojUEpHiwJ3API9j8kL6bzrzgPvc1/2Buek3CFH/A7aq6utp5hW6cyEi\nFVOfCBGRUsCfcCpPlwO3u6sVinOhqk+qak1VrYtzfVimqv0ohOdCREq7d86IyCXATcAm8vA/EpDt\nDNyxDl7n9wZpL3ockl+JyFQgCqgAHAbG4GT8GUANYB9wu6oe9ypGfxCRVsCXOH/c6k5P4rRYn07h\nOhcNcSoCw9zpI1V9XkTq4DxkcSmwHujnPnhRKIhIO2CEqnYrjOfCfc+zcf43igJTVPVFESlPLv9H\nAjIZGGOM8a9ALCYyxhjjZ5YMjDHGWDIwxhhjycAYYwyWDIwxxmDJwBhjDJYMTIgRkUoiMkVEdrl9\ntXwtIp50dCgi7UTkhjS/DxKRfl7EYkx2ArFvImPyYw4wSVXvBhCRGkA3Xx1MRIqoanImi6OAU8Aq\nAFWd6Ks4jMkva3RmQoaIdACeUdX2GSwLA14E2uF0AT1BVd92W7BGA0eAa4C1qnqPu811wCvAJe7y\n+1T1sIgsB77H6SPnQ+AH4GmgGHAUuBsojdPD6jkgFhiC04VEvKq+IiLXAv8GSuH0PjpQVU+4+/4G\np2vqssD9qvp1gZ4oYzJgxUQmlFwNrMtk2f043T1fj9OR14MiUstddi0wFGgAXC4iLd0O8sYDvVS1\nGc6AQ39Ps79iqtpcVV8FVqhqC1X9I/AR8Liq7gXeAl5V1esyuKC/CzymqtcCm3G6HElVxI3zEZxE\nZYzPWTGRCVki8ibQGkgE9gINRSS1I7MywB+AJOBbVT3obvM9UBs4gXOnsFhEUgcPSdsL5kdpXtcQ\nkelAVZy7g5+yiasMUNYdxAicxDA9zSofuz+/A2phjB9YMjChZAvQK/UXVR3sdtj1HU4yGKKqi9Nu\n4BYTJaSZlYzzfyHAZlVtlcmxTqd5PR54WVU/cfc3JpNtLjh0FstS40mNxRifs2IiEzJUdRlQQkQG\npZkdjjsSFPBw6iDpIvIHESmdxe52AJEi0sJdv6iINMhk3TL8ftfQP838eHdZ+jhPAnFur6wA9+CM\nT5CRrJKGMQXGvnWYUNMdeE1EHsepuD2NU4Y/0+3ud51b7PMrGY8LqwCqmiQivYHx7jgCRXAGUNnK\nxaNIjQVmikgczuDjtd3589353XAqkNNudx/wljs2wY/AgLTHTx+PMb5mTxMZY4yxYiJjjDGWDIwx\nxmDJwBhjDJYMjDHGYMnAGGMMlgyMMcZgycAYYwyWDIwxxgD/D9+y/f19p4HMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7eff90145128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot loss over time\n",
    "plt.plot(loss_vec_l1, 'k-', label='L1 Loss')\n",
    "plt.plot(loss_vec_l2, 'r--', label='L2 Loss')\n",
    "plt.title('L1 and L2 Loss per Generation')\n",
    "plt.xlabel('Generation')\n",
    "plt.ylabel('L1 Loss')\n",
    "plt.legend(loc='upper right')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
